Co-streaming within a live interactive video game streaming service

ABSTRACT

Technologies are described for enabling co-streaming of live video game streams by multiple broadcasters. For example, broadcasters that are eligible to co-stream can be determined. Invitations can be sent to the broadcasters to join a co-stream. After acceptance, a co-stream can be created. The co-stream can be streamed to viewer devices.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.15/628,512, filed Jun. 20, 2017, which claims the benefit of U.S.Provisional Application No. 62/482,537, filed Apr. 6, 2017, which arehereby incorporated by reference.

BACKGROUND

Video game streaming services provide live video game streaming toviewers via the Internet. With some game streaming services, the viewerscan view the gameplay of one of the users playing a video game, and canswitch to watch different users playing the video game. However, theability to stream or watch the gameplay of more than one video gameplayer at a time is limited. For example, some services allow the viewerto watch multiple separate game streams on the same computer display.However, such solutions do not provide an integrated experience and donot allow the broadcasters and the game streaming service to managemultiple game streams.

Therefore, there exists ample opportunity for improvement intechnologies related to video game streaming services.

SUMMARY

This Summary is provided to introduce a selection of concepts in asimplified form that are further described below in the DetailedDescription. This Summary is not intended to identify key features oressential features of the claimed subject matter, nor is it intended tobe used to limit the scope of the claimed subject matter.

Technologies are described for enabling co-streaming within a live videogame streaming service. For example, broadcasters that are eligible toco-stream can be automatically determined. Invitations to thebroadcasters can be automatically sent. After acceptance, a co-streamcan be created. The co-stream can be streamed to viewer devices fordisplay to viewers of the co-stream.

As another example, a request can be received from a first broadcasterto invite a second broadcaster to join a co-stream. In response to therequest, an invitation can be sent to the second broadcaster to join theco-stream. After acceptance, a co-stream can be created. The co-streamcan be streamed to viewer devices.

As another example, users that are eligible to co-stream can beautomatically determined. Invitations to the users can be automaticallysent. After acceptance, a co-stream can be created. The co-stream can bestreamed to viewer devices.

As described herein, a variety of other features and elements can beincorporated into the technologies separately and/or in combination.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is diagram depicting an example live video game streamingenvironment.

FIG. 2 is diagram depicting an example live video game streamingenvironment supporting co-streaming by broadcasters.

FIG. 3 is diagram depicting an example live video game streamingenvironment supporting co-streaming with independent broadcasterstreams.

FIG. 4 is diagram depicting an example live video game streamingenvironment supporting co-streaming with a single composited video gamestream.

FIG. 5 is a flowchart of an example method for co-streaming within alive interactive video game streaming service.

FIG. 6 is a flowchart of an example method for co-streaming within alive interactive video game streaming service using explicitinvitations.

FIG. 7 is a flowchart of an example method for co-streaming within alive interactive video game streaming service.

FIG. 8 is a diagram of an example computing system in which somedescribed embodiments can be implemented.

FIG. 9 is an example mobile device that can be used in conjunction withthe technologies described herein.

DETAILED DESCRIPTION Overview

As described herein, various technologies are described for enablinginteractivity with video games by viewers of a live video game streamingservice. The live video game streaming service is a service in which abroadcaster plays a video game (e.g., on the broadcasters computingdevice) which is then streamed live to a number of viewers (e.g., tens,hundreds, thousands, or more). The live video game can be streamed by astreaming service (e.g., a cloud-based streaming service operating as aweb site). The viewers of the live video game stream can interact withthe video game, and control aspects of gameplay, in a variety of ways.For example, the viewers can drop items in the game (e.g., weapons,armor, power-ups, etc.), control choices (e.g., tell the broadcasterwhich direction to go, or which action to take, in the game), takecontrol of the broadcaster's avatar and/or other game characters, etc.One example of a live video game streaming service is the Mixerinteractive livestreaming service provided by Microsoft® (previously theBeam service).

In order to support the live interactivity, the live video gamestreaming service operates with very low latency (e.g., sub-secondlatency, such as much lower than one second). Operating with very lowlatency allows the viewers to interact in real-time with the video gameas the broadcaster is playing, and have their interactions occur insynchronization with the gameplay. For example, if the broadcaster isplaying a game in which their avatar has a choice of taking paths to theleft or the right, then a viewer could activate a command to direct thebroadcaster to take one of the paths in real-time. Operating with verylow latency makes the interaction effective and enhances the interactionexperience of the viewers (e.g., with higher latency if the broadcasterhad already taken one of the paths then the viewer's interaction wouldnot work). Sub-second latency allows the broadcaster to communicate inreal-time, and the viewers to respond in real-time. Operating withSub-second latency also provides very close synchronization (e.g., lessthan one-second difference) between multiple broadcaster streams in aco-streaming situation. For example, if two broadcasters areco-streaming the same video game in the same multiplayer session,viewers will observe the two broadcaster's streams as very closelysynchronized which will result in a more seamless experience (e.g.,instead of having broadcaster streams that are not closely synchronized,such as with greater than one second difference, which can result in aconfusing experience for the viewers).

Various types of interactivity can be provided within the live videogame streaming service environment. For example, the viewers can beprovided with interactive controls (e.g., buttons, joysticks, gamepadcontrols, or other interactive controls) that can be activated to affectthe video game, such as spawning enemies in the video game, droppingpower-ups, creating in-game items (e.g., weapons, armor, etc.),increasing player health, taking control of the broadcaster's player inthe game, and taking control of another character in the game. Examplesof interactive controls include buttons that the viewers can press andjoysticks or directional controls that the user can move in variousdirections.

Live Video Game Streaming Environment

In the technologies described herein, an environment can be provided tosupport live video game streaming. For example, various computingdevices and computing services can provide a live interactive video gamestreaming service web site that communicates with broadcasters toreceive interactive video game streams and communicates with viewers toprovide the interactive video game streams.

FIG. 1 is a diagram 100 depicting an example live video game streamingenvironment. In the diagram 100, a live interactive video game streamingservice 110 provides live interactive video game streaming allowingviewers using viewer devices 130 to view and interact with video gamesbeing played by broadcasters using broadcaster devices 120. The liveinteractive video game streaming service 110 can include various typesof computer hardware and software (e.g., servers, web hosting software,streaming software, web server software, cloud computing resources,etc.). In some implementations, the live interactive video gamestreaming service 110 operates as a web site where the viewers canbrowse various live video game streams, view a selected live video gamestream, and interact in real-time with the video game being played.

In the diagram 100, a broadcaster device 120 is used by a broadcaster tolive stream a video game. The broadcaster device 120 can be a desktopcomputer, laptop computer, or another type of computing device.

Using the broadcaster device 120, the broadcaster plays the video gameand streams it live to the viewer devices 130 via the live interactivevideo game streaming service 110. In some implementations, the videogame is running locally on the broadcaster device 120, as indicated bythe video game engine 124. In some implementations, running the videogame involves communication by the broadcaster device 120 with anexternal or remote video game service 140 (e.g., when playing an onlineand/or multiplayer video game). In some implementations, the video gameis running on a computing device (not depicted) that is separate fromthe broadcaster device 120, such as a gaming console. For example, thebroadcaster device 120 can obtain the game stream from the separatecomputing device (e.g., from the gaming console) via a network or avideo capture device.

A game interaction manager (e.g., game interaction manager 122, 126,and/or 142) manages interaction with the video game. For example, thegame interaction manager can maintain the control state of the variousinteractive controls (e.g., received as control events from the viewerdevices 130 via the live interactive video game streaming service 110),communicate using a protocol that communicates control events and thatsupports switching communication modes, and/or supports co-streamingamong a number of broadcasters. The game interaction manager 122 can runon the broadcaster device 120 (e.g., as part of a software component orsoftware development kit that is associated with the live interactivevideo game streaming service 110). The game interaction manager 126 canalso run as a part of the video game engine 124. The game interactionmanager 142 can also run as part of the game service 140. In someimplementations, only one or more instances of the game interactionmanager are present (e.g., just game interaction manager 122). In someimplementations, the interaction manager is called the Mixer manager.

Co-Streaming

In the technologies described herein, separate live interactive videogame streams can be provided for display to viewers. For example,multiple broadcasters (e.g., multiple broadcaster devices, one of whichcould be broadcaster device 120) can be separately broadcasting liveinteractive video game streams. The video game streams from the multiplebroadcasters can be provided (e.g., delivered individually or combinedat the live interactive video game streaming service 110) for display toviewers (e.g., to viewer devices 130). The video game streams providedby the broadcasters may or may not be interactive. For example, aco-stream can be created in which some or all of the broadcaster videogame streams are not interactive (e.g., the broadcasters can bestreaming their gameplay with no interactivity exposed to viewers).

For example, two broadcasters may be live streaming their video games inthe same multiplayer session of a multiplayer game (e.g., an onlinemultiplayer game such as Halo). One of the broadcasters can invite theother broadcaster to co-stream. Upon accepting, both broadcast streamscan be provided to viewers (e.g., shown side-by-side or in anotherarrangement) by the video game streaming service. For example, the liveinteractive video game streaming service 110 can create a co-streamcomprising individual broadcaster streams that are transmitted anddisplayed side-by-side in the displayed game stream 132 windows of theviewer devices 130. In some implementations, all of the viewers thatwere watching either of the two broadcasters (e.g., that were subscribedto either of the two broadcasters' channels) will automatically receivethe co-stream.

In some implementations, social features will also be combined when aco-stream is created. For example, the chat areas (e.g., chat area 136)associated with two broadcasters' streams can be combined when the twobroadcasters are co-streaming.

While the technologies for co-streaming are described in the context ofa live interactive video game streaming service, co-streaming can beperformed within other types of video streaming services that are notbroadcasting video game content. For example, the live interactive videogame streaming service 110 can be another type of video streamingservice, such as a video streaming service providing creative content (eg, making music, web shows, programming, painting, etc.).

Co-Streaming with Explicit Invitations

In some implementations, co-streaming is performed using explicitinvitations. For example, one broadcaster can explicitly invite one ormore other broadcasters (and the one or more other broadcaster canaccept or decline). If accepted, the video game streams from thebroadcaster sending the invitation as well as the one or more otherbroadcasters accepting the invitation can be streamed to the viewers fordisplay as a co-stream. The co-stream can provide multiple broadcastervideo game streams to the viewers as independent streams and/or as acombined stream (e.g., a combined stream that is created as a singlecomposited stream).

FIG. 2 is diagram 200 depicting an example live video game streamingenvironment supporting co-streaming by broadcasters. In the diagram 200,broadcaster device 120 and broadcaster device 230 are streaming a liveinteractive video game stream. Specifically, broadcaster device 120 isstreaming the video game stream indicated at 220, and broadcaster device230 is streaming the video game stream indicated at 225. The individualbroadcaster's video game streams 220 and 225 can be co-streamed to theviewer devices 130. In this example, both the video game stream 220 andthe video game stream 225 are displayed at the viewer devices 130 as adisplayed co-stream 240, which displays both streams side-by-side(although other display arrangements are supported). While the diagram200 depicts two broadcasters, the co-streaming technology can supportany number of broadcasters. For example, three, four, or morebroadcasters' live interactive video game streams can be co-streamed toviewers (e.g., as independent and/or combined streams).

In some implementations, the co-stream is initiated using an explicitinvitation (also called a manual invitation). For example, thebroadcaster using broadcaster device 120 can invite the broadcasterusing broadcaster device 230 to join in a co-stream. The explicitinvitation can be initiated using a graphical user interface control(e.g., a menu that allows the broadcaster using broadcaster device 120to select the broadcaster using broadcaster device 230) or using anothertype of control (e.g., sending an instant message or email). Therecipient of the explicit invitation (in this example, the broadcasterusing broadcaster device 230) can be notified of the invitation toco-stream (e.g., as a pop-up or alert message presented on thebroadcaster's display). The recipient can accept or deny the invitationto co-stream (e.g., select a graphical control to accept, or decline,the invitation).

The live interactive video game streaming service 110 performsoperations supporting co-streaming, as depicted at 210. For example, thelive interactive video game streaming service 110 can receive a requestfrom broadcaster device 120 to invite the broadcaster using broadcasterdevice 230 to co-stream. If the live interactive video game streamingservice 110 receives an acceptance from the broadcaster device 230, thelive interactive video game streaming service 110 can create a co-streamand send it to the viewer devices 130.

An explicit invitation can be sent to and/or from a user that iscurrently playing and streaming a game, a user that is currently playinga game but not streaming, and/or a user that is not playing a game. Forexample, a user that is currently playing and broadcasting a game caninvite their friend (who is not currently playing) to start playing thegame and join in co-streaming the game.

Co-Streaming with Automatic Invitations

In some implementations, invitations to join a co-stream areautomatically provided (also called an implicit invitation). Forexample, the live interactive video game streaming service can send aninvite to one or more broadcasters to create a co-stream. For example,the live interactive video game streaming service can automaticallydetermine that two or more broadcasters are currently streaming in thesame multiplayer session of the same multiplayer game and send out aninvite to the broadcasters to co-stream.

The co-streaming depicted in the diagram 200 of FIG. 2 can also beinitiated using an automatic invitation. As depicted at 210, the liveinteractive video game streaming service 110 can perform operations toautomatically initiate co-streaming invitations. For example, the liveinteractive video game streaming service 110 can determine thatbroadcaster devices 120 and 230 are both streaming in the samemulti-player game. In response, the live interactive video gamestreaming service 110 can automatically send invitations to thebroadcasters using broadcaster devices 120 and 230 to join a co-stream.The invitations can take the form of a pop-up or alert message presentedon the broadcasters' displays. If the broadcasters accept, the liveinteractive video game streaming service 110 can create a co-stream andsend it to the viewer devices 130.

The live interactive video game streaming service 110 can automaticallydetermine to invite one or more broadcasters to join in a co-streambased on a variety of different criteria, which can be appliedseparately or in combination. One example criteria is the video gamethat the broadcasters are playing. For example, if multiple broadcastersare playing the same multiplayer video game in the same multiplayersession, then the live interactive video game streaming service 110 canautomatically determine that the multiple broadcasters should be invitedto co-stream.

Another example criteria is whether the broadcasters are in a partytogether. For example, if multiple broadcasters are playing a video gameand are in the same party, then the live interactive video gamestreaming service 110 can automatically determine that the multiplebroadcasters should be invited to co-stream. The broadcasters that arein the same party together may be playing the same game or differentgames. In addition, a user in a party that is not playing a game maystill be invited to co-stream. For example, a user may providecommentary and therefore join the co-stream as a commentary broadcaster(e.g., with audio and video, or just audio).

Another example criteria is whether the broadcasters are performing asimilar activity. For example, if two broadcasters are demonstrating aspeed run of the same game, then the two broadcasters can beautomatically invited by the live interactive video game streamingservice 110 to co-stream their speed runs. The broadcasters can beplaying different games or the same game (e.g., performing speed runsfor the same single-player game).

Another example criteria is whether the broadcasters are playing in atournament together. For example, multiple broadcasters that are playingin the same tournament can be invited to co-stream. For example aspecific broadcaster from each team in the tournament can be selected toco-stream (e.g., the top player from each team).

Another example criteria are the social links between users. Forexample, broadcasters that are friends within a social network can beinvited to join a co-stream.

An automatic invitation can be sent to a user that is currently playinga game and currently steaming the game. For example, automaticinvitations can be sent to multiple users that are currentlybroadcasting in the same multiplayer game session. The users can receivea message asking them to join a co-stream.

An automatic invitation can be sent to a user that is currently playinga game but not currently streaming the game. For example, an automaticinvitation can be sent to a user that is currently playing (but notstreaming) in a multiplayer game session with another user that iscurrently broadcasting in the same multiplayer game session. The userthat is currently playing can receive a request to start streaming andjoin the co-stream. The user that is currently broadcasting can receivea request to join the co-stream.

An automatic invitation can be sent to a user that is not currentlyplaying a game. For example, the user may be in a party with anotheruser that is currently playing and/or broadcasting in a game. The userthat is not currently playing can receive an invitation to start playingand broadcasting the game, and join a co-stream with the other user.

In general, any of the users that are being invited to join theco-stream can be playing a game and streaming, playing a game but notstreaming, or not playing a game.

Acceptance of Co-Streaming Invitations

In some implementations, a user is notified of a co-streaming invitation(an explicit or automatic invitation) and has the option to accept ordecline. For example, a message or alert can be presented to the userwhile the user is playing and/or broadcasting a video game asking if theuser wants to join a co-stream. Additional information can also bedisplayed in association with the invitation (e.g., the identity of theother broadcaster(s) in the co-stream, the identity of the user sendingthe invitation if it is an explicit invitation, an indication that theinvitation was automatically generated, the video game being played,etc.). The user receiving the invitation then has the option to acceptthe invitation (e.g., select an accept button) or decline (e.g., selecta decline button).

In some implementations, acceptance of co-streaming invitation can alsobe controlled by as user-configurable setting (a setting or other typeof preference that the user can set to control co-streaming acceptance).For example, a user can configure a setting to automatically acceptco-streaming invitations from the user's friends or party members (e.g.,and automatically decline co-streaming invitations from users that arenot friends or party members). In this situation, if the user receivesan invitation from the user's friend, then the user can be automaticallyjoined in the co-stream without the user having to accept (however, theuser can be presented with a message indicating that the user has beenautomatically joined in the co-stream). Acceptance of co-streaminginvitations can also be controlled by an opt-in user-configurablesetting. For example, users that are playing in a tournament can opt-into allow co-streaming during the tournament without the users having totake any affirmative action to accept co-streaming invitations duringthe tournament. Acceptance based on user-configurable settings can beused for acceptance of an explicit invitation and/or an automaticinvitation.

Virtual Cameras

In some implementations, co-streams can include virtual cameras. Avirtual camera presents a stream (comprising video and/or audio) of thevideo game from a perspective other than a broadcaster (i.e., other thana user streaming the video game that the user is playing). A virtualcamera can be created automatically (e.g., upon detecting an in-gameevent, such as a boss spawn) and automatically joined to a co-stream(e.g., by the live interactive video game streaming service 110).

A virtual camera can be used to present video and/or audio of an in-gameevent. For example, if a number of broadcasters are co-streaming in thesame multiplayer session and an in-game event occurs (e.g., a bosscreature spawns), then a virtual camera can be created and joined as aseparate stream of the co-stream to depict the gameplay from the pointof view of the in-game event (e.g., from the point of view of the bosscreature).

Combined Social Features

In some implementations, social features are combined when multiplebroadcasters are co-streaming One social feature that can be combined isthe chat area (also called a chat panel). Without co-streaming, a chatarea displays viewer chat information for the viewers that are viewing aparticular broadcaster's live video game stream. When co-streamingbegins, the viewers of each individual broadcaster are associated withthe co-stream. As a result of the association with the co-stream, allviewers of the co-stream will see a combined chat area (also called aunified chat panel) that displays chat information from all of theviewers of the co-stream (all the viewers that were previously watchingthe separate broadcaster streams before the co-stream began). Forexample, the chat area 136 can represent a combined chat area displayingchat information from viewers of the displayed co-stream 240.

The combined chat area can display chat information from the viewers ina seamless manner. For example, the chats from the viewers can bedisplayed in order based on the time the chats are submitted withoutseparating them based on the original broadcaster a given viewer wasassociated with (e.g., based on the original broadcaster's channel thegiven user was watching).

Providing a combined chat experience can be performed automatically andseamlessly from the point of view of the viewers. For example, aspecific viewer could be viewing the live interactive video game stream220 from broadcaster device 120 (before co-streaming has started). Oncethe co-streaming has begun, the specific viewer can automaticallyreceive the displayed co-stream 240, and the chat area 136 canautomatically start displaying chat information from all of the viewersof the co-stream (in this example, the viewers previously viewing videogame stream 220 as well as the viewers previously viewing video gamestream 225).

Another social feature that can be combined is a poll. For example, theviewers of all of the broadcasters joined in the co-stream canparticipate in combined polls. Another combined social feature is anoverall viewer count that combines the viewer counts from allbroadcaster streams that are included in the co-stream. The overallviewer count can be displayed to the viewers (e.g., associated with thechat area 136).

Another social feature that can be combined is moderation. A givenbroadcaster can have one or more moderators that are associated with thebroadcaster's video game stream. The moderators perform moderationactions for their associated stream (e.g., banning, timeouts, removingor editing chat posts, answering questions, etc.). When a co-stream iscreated, the moderators associated with the broadcasters of theco-stream can perform moderation actions across all of the viewers ofthe co-stream. For example, a moderator associated with a firstbroadcaster can ban a viewer that is associated with a secondbroadcaster. However, in some implementations, a moderator associatedwith a specific broadcaster cannot permanently affect the viewersassociated with a different broadcaster. For example, the moderationactions performed by a moderator associated with a first broadcaster ona viewer associated with a different broadcaster may only last while theco-stream is active (e.g., if the moderator bans a viewer associatedwith a different broadcaster, then the viewer will only be banned whilethe co-stream is active, and would not be banned from the differentbroadcaster's stream after the co-stream ends).

The viewers of a co-stream can remain associated with their initialbroadcaster's stream (e.g., associated with the initial broadcaster'schannel). For example, a particular viewer can be viewing the video gamestream 220 before the co-stream is created. Once the co-stream iscreated, the particular viewer would see the displayed co-stream 240comprising both broadcaster's video game streams 220 and 225. Once theco-stream ends, the particular viewer would return to just viewing thevideo game stream 220 (to the original broadcaster that the viewer wasviewing before the co-stream began).

Combined Interactive Controls

In some implementations, interactive controls are combined when multiplebroadcasters are co-streaming. For example, some or all of theinteractive controls (e.g., interactive controls 134) can be combinedinteractive controls during co-streaming. Combined interactive controlsaffect multiple broadcasters. An example of a combined interactivecontrol is an interactive control that sets the spawn point for multiplebroadcasters in a video game. Another example of a combined interactivecontrol is an interactive control that drops an in-game item (e.g., apower-up) for multiple broadcasters.

Using combined interactive controls unifies interactivity of the viewersof the co-stream. For example, instead of each broadcaster's usershaving separate interactive controls, combined interactive controlsallow all viewers to affect gameplay across the broadcasters of theco-stream.

Viewer Selection of Broadcasters

In some implementations, viewers can select the broadcasters that theywant to view. For example, a particular viewer can be presented with alist of broadcasters and the viewer can select one or more of them toview in a co-stream.

In some implementations, viewers of a video game event can select whichof the broadcasters they want to view. For example, a video game eventcould be a five versus five tournament. Each viewer of the match canindependently select one or more of the ten broadcasters to view in aco-stream. For example, a specific viewer could select two specificbroadcasters on the first team and two specific broadcasters on thesecond team to view in a co-stream. In some implementations, the totalnumber of broadcasters supported in a co-stream is limited (e.g., due tocomputing resource or bandwidth limitations), such as to a maximum offour broadcasters.

Live Interactive Video Game Streams

In order to provide a co-stream experience for the viewers, a liveinteractive video game stream from the broadcasters is delivered to theclient devices. For example, the live interactive video game streamingservice can receive the individual broadcasters' streams and transmitthem (e.g., as independent streams and/or as combined streams) to theviewer devices.

In some implementations, the co-stream comprises independent video gamestreams from each of the broadcasters of the co-stream. Includingindependent video game streams provides a number of advantages andsupports additional functionality. One advantage and feature is that theindependent video game streams can be saved and accessed later (e.g.,each broadcaster's stream can be saved as a separate video file with anassociation to the co-stream of which it was a part). For example, aviewer could view a co-stream at a later time with full control overwhat the viewer wants to see. For example, the viewer could select whichbroadcaster's stream to view (e.g., just one broadcaster's stream, or ingeneral any one or more of the available broadcaster streams in theco-stream). The viewer could re-view the co-stream from a differentperspective (e.g., view a different broadcaster's stream), or switchbetween broadcasters while viewing the co-stream.

Another advantage and feature is that the independent video game streamscan be edited independently. For example, a user could access apreviously saved co-stream and create an edited version. The editedversion could use clips from one or more of the independent broadcasterstreams of the co-stream. For example, the user could create ahighlights video using segments from one or more of the individualstreams of the co-stream.

Another advantage and feature is that the service and/or viewers candynamically change the bitrate of the streams on a viewer-by-viewerbasis. For example, if a particular viewer device has limited bandwidth,then the live interactive video game streaming service can independentlyadjust the bitrates of the broadcaster streams (e.g., lower the bitrateof all streams or just specific streams). The bitrate adjustment canalso be controlled by which streams a particular viewer is currentlyviewing. For example, if a particular viewer is currently viewing threeout of four broadcaster streams of a co-stream, then the three currentlyviewed streams can be sent at a higher bitrate than the fourth stream.

Another advantage and feature is that the viewers can individuallyselect which broadcasters to view in the co-stream and switch betweenthem. For example, if a viewer is watching a 16 versus 16 video gametournament, the viewer could select four of the 32 players to watch in aco-stream. The viewer could switch to watch different broadcasters atany time.

Another advantage and feature is that computing resource savings can berealized using independent video game streams. For example, the liveinteractive video game streaming service only needs to perform minimalprocessing to deliver the independent streams (e.g., no decoding,compositing, or re-encoding is needed), which can save computerprocessor and memory resources over solutions that process the streams(e.g., creating a composited video game stream). In addition, differentviewers can be provided co-streams with different broadcasters,different bitrates, etc., with minimal additional load being placed onthe live interactive video game streaming service.

FIG. 3 is diagram 300 depicting an example live video game streamingenvironment supporting co-streaming with independent broadcasterstreams. As depicted in the diagram 300, the live interactive video gamestreaming service 110 receives broadcaster video game streams 220 and225. The live interactive video game streaming service 110 creates aco-stream comprising independent video game streams from eachbroadcaster of the co-stream, as depicted at 310. The live interactivevideo game streaming service 110 also saves the independent video gamestreams from each broadcaster of the co-stream (e.g., as separate videofiles), as depicted at 310. The live interactive video game streamingservice 110 transmits the co-stream to the viewer devices 130, asdepicted at 320. Because the co-stream contains independently accessiblebroadcaster streams, the viewer devices 130 can control how the streamsare displayed. In this example, in the displayed co-stream 240, one ofthe broadcaster streams has been resized by a viewer to display at alarger size than the other broadcaster stream. For example, the viewerdevices 130 can receive the independent broadcaster steams in theco-stream (as depicted at 320), perform various types of processing(e.g., re-sizing, layout arrangement, compositing, etc.), and displaythe co-stream (e.g., as depicted at 240).

In some implementations, the co-stream is delivered to the viewers as acombined live interactive video game stream that is created as a singlecomposited video stream. In order to create the single composited videostream, the individual broadcaster video game streams can be received,decoded, composited or otherwise combined into a single video, andre-encoded into a single composited video stream. The format or layoutof the individual broadcaster video game streams is fixed when thesingle composited video stream is created (e.g., the size and/orarrangement of the broadcaster videos is set at the time the individualstreams are composited or otherwise combined). For example, the layoutcould be two broadcasters displayed side-by-side, or four broadcastersdisplayed each in one-quarter of the display area. When using a singlecomposited video stream, the live interactive video game streamingservice can still save the independent broadcaster streams before theyare composited or otherwise combined, which can support some of theindependent stream features (e.g., allow viewers to view or editindependent streams at a later time).

FIG. 4 is diagram 400 depicting an example live video game streamingenvironment supporting co-streaming with a single composited video gamestream. As depicted in the diagram 400, the live interactive video gamestreaming service 110 receives broadcaster video game streams 220 and225. The live interactive video game streaming service 110 creates asingle composited video game stream comprising the video game streamsfrom each broadcaster of the co-stream, as depicted at 410. The liveinteractive video game streaming service 110 transmits thecombined-co-stream with the single composited video stream to the viewerdevices 130, as depicted at 420.

Co-Streaming Methods

In the technologies described herein, methods can be provided forco-streaming within a live interactive video game streaming service. Forexample, the live interactive video game streaming service can identifyusers (e.g., users that are not currently playing a video game, usersthat are currently playing a video game but not broadcasting, and/orusers that are currently playing a video game and broadcasting toviewers) and invite them to co-stream. Invitations can be sent (e.g.,explicitly by other users and/or automatically by the service) to usersto co-stream. Upon acceptance, a co-stream can be created comprising thevideo game streams of the broadcasters joined in the co-stream. Theco-stream can be streamed to viewer devices where the co-stream can bedisplayed to viewers using the viewer devices.

FIG. 5 is a flowchart of an example method 500 for co-streaming within alive interactive video game streaming service. For example, the examplemethod 500 can be performed by the live interactive video game streamingservice 110.

At 510, an automatic determination is made that two or more broadcastersare eligible to co-stream. For example, the broadcasters can be playingthe same video game in the same multi-player session.

At 520, invitations are automatically sent to the two or morebroadcasters that were determined at 510. The invitations can beaccepted or declined. For example, a broadcaster can receive a request(e.g., displayed as an alert or pop-up message on the broadcaster'scomputer display) and indicate whether the broadcaster accepts ordeclines the invitation. In some implementations, a user-configurablesetting is used to automatically determine whether the broadcasteraccepts or declines without the invitation being displayed to thebroadcaster.

At 530, after acceptance by the two or more broadcasters (e.g., explicitacceptance or acceptance based on user-configurable settings), aco-stream is created. The co-stream can comprise independent broadcastervideo game streams or a combined stream (e.g., a single composited videogame stream).

At 540, the co-stream is streamed (e.g., via the Internet) to aplurality of viewer devices as a co-stream. The plurality of viewerdevices provide the co-stream to viewers that are associated with thetwo or more broadcasters (e.g., viewers that were watching broadcastchannels of the two or more broadcasters before the co-stream began).

FIG. 6 is a flowchart of an example method 600 for co-streaming within alive interactive video game streaming service using explicitinvitations. For example, the example method 600 can be performed by thelive interactive video game streaming service 110.

At 610, a request is received from a first broadcaster to invite asecond broadcaster to join a co-stream. In some implementations, thefirst broadcaster is currently steaming a video game. The secondbroadcaster may or may not be currently streaming. In someimplementations, the first broadcaster is playing a video game (but notbroadcasting). In some implementations, the first broadcaster is notplaying a video game.

At 620, in response to receiving the request, an invitation is sent tothe second broadcaster to join the co-stream with the first broadcaster.For example, the invitation can indicate the identity of the firstbroadcaster and request that the second broadcaster join a co-stream.The invitation can also include the identity of the video game that willbe played, the multi-player session, etc.

At 630, after acceptance by the second broadcaster, a co-stream iscreated. The co-stream can comprise independent broadcaster video gamestreams or a combined stream (e.g., a single composited video gamestream).

At 640, the co-stream is streamed (e.g., via the Internet) to aplurality of viewer devices. The plurality of viewer devices provide theco-stream to viewers that are associated with the first and secondbroadcasters (e.g., viewers that were watching broadcast channels of thefirst and second broadcasters).

The example method 600 can be used to explicitly invite additionalbroadcasters. For example, a broadcaster associated with the co-streamcan invite a number of additional broadcasters to join the co-stream.

FIG. 7 is a flowchart of an example method 700 for co-streaming within alive interactive video game streaming service. For example, the examplemethod 700 can be performed by the live interactive video game streamingservice 110.

At 710, an automatic determination is made that two or more users areeligible to co-stream. In some implementations, the users are currentlyplaying a video game (e.g., the same video game). However, in someimplementations, a user may be currently playing a video game andbroadcasting a live interactive video game stream, playing a video gamebut not broadcasting, or not playing a video game.

At 720, invitations are automatically sent to the two or more users thatwere determined at 710. The invitations can be accepted or declined. Forexample, a user can receive a request (e.g., displayed as an alert orpop-up message on the user's computer display) and indicate whether theuser accepts or declines the invitation. In some implementations, auser-configurable setting is used to automatically determine whether theuser accepts or declines without the invitation being displayed to theuser.

At 730, after acceptance by the two or more users (e.g., explicitacceptance or acceptance based on user-configurable settings), aco-stream is created. The co-stream can comprise independent broadcastervideo game streams or a combined stream (e.g., a single composited videogame stream).

At 740, the co-stream is streamed (e.g., via the Internet) to aplurality of viewer devices. The plurality of viewer devices provide theco-stream to viewers that are associated with the two or more users(e.g., viewers that were subscribed to broadcast channels of the two ormore users).

Computing Systems

FIG. 8 depicts a generalized example of a suitable computing system 800in which the described innovations may be implemented. The computingsystem 800 is not intended to suggest any limitation as to scope of useor functionality, as the innovations may be implemented in diversegeneral-purpose or special-purpose computing systems.

With reference to FIG. 8, the computing system 800 includes one or moreprocessing units 810, 815 and memory 820, 825. In FIG. 8, this basicconfiguration 830 is included within a dashed line. The processing units810, 815 execute computer-executable instructions. A processing unit canbe a general-purpose central processing unit (CPU), processor in anapplication-specific integrated circuit (ASIC), or any other type ofprocessor. In a multi-processing system, multiple processing unitsexecute computer-executable instructions to increase processing power.For example, FIG. 8 shows a central processing unit 810 as well as agraphics processing unit or co-processing unit 815. The tangible memory820, 825 may be volatile memory (e.g., registers, cache, RAM),non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or somecombination of the two, accessible by the processing unit(s). The memory820, 825 stores software 880 implementing one or more innovationsdescribed herein, in the form of computer-executable instructionssuitable for execution by the processing unit(s).

A computing system may have additional features. For example, thecomputing system 800 includes storage 840, one or more input devices850, one or more output devices 860, and one or more communicationconnections 870. An interconnection mechanism (not shown) such as a bus,controller, or network interconnects the components of the computingsystem 800. Typically, operating system software (not shown) provides anoperating environment for other software executing in the computingsystem 800, and coordinates activities of the components of thecomputing system 800.

The tangible storage 840 may be removable or non-removable, and includesmagnetic disks, magnetic tapes or cassettes, CD-ROMs, DVDs, or any othermedium which can be used to store information and which can be accessedwithin the computing system 800. The storage 840 stores instructions forthe software 880 implementing one or more innovations described herein.

The input device(s) 850 may be a touch input device such as a keyboard,mouse, pen, or trackball, a voice input device, a scanning device, oranother device that provides input to the computing system 800. Forvideo encoding, the input device(s) 850 may be a camera, video card, TVtuner card, or similar device that accepts video input in analog ordigital form, or a CD-ROM or CD-RW that reads video samples into thecomputing system 800. The output device(s) 860 may be a display,printer, speaker, CD-writer, or another device that provides output fromthe computing system 800.

The communication connection(s) 870 enable communication over acommunication medium to another computing entity. The communicationmedium conveys information such as computer-executable instructions,audio or video input or output, or other data in a modulated datasignal. A modulated data signal is a signal that has one or more of itscharacteristics set or changed in such a manner as to encode informationin the signal. By way of example, and not limitation, communicationmedia can use an electrical, optical, RF, or other carrier.

The innovations can be described in the general context ofcomputer-executable instructions, such as those included in programmodules, being executed in a computing system on a target real orvirtual processor. Generally, program modules include routines,programs, libraries, objects, classes, components, data structures, etc.that perform particular tasks or implement particular abstract datatypes. The functionality of the program modules may be combined or splitbetween program modules as desired in various embodiments.Computer-executable instructions for program modules may be executedwithin a local or distributed computing system.

The terms “system” and “device” are used interchangeably herein. Unlessthe context clearly indicates otherwise, neither term implies anylimitation on a type of computing system or computing device. Ingeneral, a computing system or computing device can be local ordistributed, and can include any combination of special-purpose hardwareand/or general-purpose hardware with software implementing thefunctionality described herein.

For the sake of presentation, the detailed description uses terms like“determine” and “use” to describe computer operations in a computingsystem. These terms are high-level abstractions for operations performedby a computer, and should not be confused with acts performed by a humanbeing. The actual computer operations corresponding to these terms varydepending on implementation.

Mobile Device

FIG. 9 is a system diagram depicting an example mobile device 900including a variety of optional hardware and software components, showngenerally at 902. Any components 902 in the mobile device cancommunicate with any other component, although not all connections areshown, for ease of illustration. The mobile device can be any of avariety of computing devices (e.g., cell phone, smartphone, handheldcomputer, Personal Digital Assistant (PDA), etc.) and can allow wirelesstwo-way communications with one or more mobile communications networks904, such as a cellular, satellite, or other network.

The illustrated mobile device 900 can include a controller or processor910 (e.g., signal processor, microprocessor, ASIC, or other control andprocessing logic circuitry) for performing such tasks as signal coding,data processing, input/output processing, power control, and/or otherfunctions. An operating system 912 can control the allocation and usageof the components 902 and support for one or more application programs914. The application programs can include common mobile computingapplications (e.g., email applications, calendars, contact managers, webbrowsers, messaging applications), or any other computing application.Functionality 913 for accessing an application store can also be usedfor acquiring and updating application programs 914.

The illustrated mobile device 900 can include memory 920. Memory 920 caninclude non-removable memory 922 and/or removable memory 924. Thenon-removable memory 922 can include RAM, ROM, flash memory, a harddisk, or other well-known memory storage technologies. The removablememory 924 can include flash memory or a Subscriber Identity Module(SIM) card, which is well known in GSM communication systems, or otherwell-known memory storage technologies, such as “smart cards.” Thememory 920 can be used for storing data and/or code for running theoperating system 912 and the applications 914. Example data can includeweb pages, text, images, sound files, video data, or other data sets tobe sent to and/or received from one or more network servers or otherdevices via one or more wired or wireless networks. The memory 920 canbe used to store a subscriber identifier, such as an InternationalMobile Subscriber Identity (IMSI), and an equipment identifier, such asan International Mobile Equipment Identifier (IMEI). Such identifierscan be transmitted to a network server to identify users and equipment.

The mobile device 900 can support one or more input devices 930, such asa touchscreen 932, microphone 934, camera 936, physical keyboard 938and/or trackball 940 and one or more output devices 950, such as aspeaker 952 and a display 954. Other possible output devices (not shown)can include piezoelectric or other haptic output devices. Some devicescan serve more than one input/output function. For example, touchscreen932 and display 954 can be combined in a single input/output device.

The input devices 930 can include a Natural User Interface (NUI). An NUIis any interface technology that enables a user to interact with adevice in a “natural” manner, free from artificial constraints imposedby input devices such as mice, keyboards, remote controls, and the like.Examples of NUI methods include those relying on speech recognition,touch and stylus recognition, gesture recognition both on screen andadjacent to the screen, air gestures, head and eye tracking, voice andspeech, vision, touch, gestures, and machine intelligence. Otherexamples of a NUI include motion gesture detection usingaccelerometers/gyroscopes, facial recognition, 3D displays, head, eye,and gaze tracking, immersive augmented reality and virtual realitysystems, all of which provide a more natural interface, as well astechnologies for sensing brain activity using electric field sensingelectrodes (EEG and related methods). Thus, in one specific example, theoperating system 912 or applications 914 can comprise speech-recognitionsoftware as part of a voice user interface that allows a user to operatethe device 900 via voice commands. Further, the device 900 can compriseinput devices and software that allows for user interaction via a user'sspatial gestures, such as detecting and interpreting gestures to provideinput to a gaming application.

A wireless modem 960 can be coupled to an antenna (not shown) and cansupport two-way communications between the processor 910 and externaldevices, as is well understood in the art. The modem 960 is showngenerically and can include a cellular modem for communicating with themobile communication network 904 and/or other radio-based modems (e.g.,Bluetooth 964 or Wi-Fi 962). The wireless modem 960 is typicallyconfigured for communication with one or more cellular networks, such asa GSM network for data and voice communications within a single cellularnetwork, between cellular networks, or between the mobile device and apublic switched telephone network (PSTN).

The mobile device can further include at least one input/output port980, a power supply 982, a satellite navigation system receiver 984,such as a Global Positioning System (GPS) receiver, an accelerometer986, and/or a physical connector 990, which can be a USB port, IEEE 1394(FireWire) port, and/or RS-232 port. The illustrated components 902 arenot required or all-inclusive, as any components can be deleted andother components can be added.

Example Implementations

Although the operations of some of the disclosed methods are describedin a particular, sequential order for convenient presentation, it shouldbe understood that this manner of description encompasses rearrangement,unless a particular ordering is required by specific language set forthbelow. For example, operations described sequentially may in some casesbe rearranged or performed concurrently. Moreover, for the sake ofsimplicity, the attached figures may not show the various ways in whichthe disclosed methods can be used in conjunction with other methods.

Any of the disclosed methods can be implemented as computer-executableinstructions or a computer program product stored on one or morecomputer-readable storage media and executed on a computing device(i.e., any available computing device, including smart phones or othermobile devices that include computing hardware). Computer-readablestorage media are tangible media that can be accessed within a computingenvironment (one or more optical media discs such as DVD or CD, volatilememory (such as DRAM or SRAM), or nonvolatile memory (such as flashmemory or hard drives)). By way of example and with reference to FIG. 8,computer-readable storage media include memory 820 and 825, and storage840. By way of example and with reference to FIG. 9, computer-readablestorage media include memory and storage 920, 922, and 924. The termcomputer-readable storage media does not include signals and carrierwaves. In addition, the term computer-readable storage media does notinclude communication connections, such as 870, 960, 962, and 964.

Any of the computer-executable instructions for implementing thedisclosed techniques as well as any data created and used duringimplementation of the disclosed embodiments can be stored on one or morecomputer-readable storage media. The computer-executable instructionscan be part of, for example, a dedicated software application or asoftware application that is accessed or downloaded via a web browser orother software application (such as a remote computing application).Such software can be executed, for example, on a single local computer(e.g., any suitable commercially available computer) or in a networkenvironment (e.g., via the Internet, a wide-area network, a local-areanetwork, a client-server network (such as a cloud computing network), orother such network) using one or more network computers.

For clarity, only certain selected aspects of the software-basedimplementations are described. Other details that are well known in theart are omitted. For example, it should be understood that the disclosedtechnology is not limited to any specific computer language or program.For instance, the disclosed technology can be implemented by softwarewritten in C++, Java, Perl, JavaScript, Adobe Flash, or any othersuitable programming language. Likewise, the disclosed technology is notlimited to any particular computer or type of hardware. Certain detailsof suitable computers and hardware are well known and need not be setforth in detail in this disclosure.

Furthermore, any of the software-based embodiments (comprising, forexample, computer-executable instructions for causing a computer toperform any of the disclosed methods) can be uploaded, downloaded, orremotely accessed through a suitable communication means. Such suitablecommunication means include, for example, the Internet, the World WideWeb, an intranet, software applications, cable (including fiber opticcable), magnetic communications, electromagnetic communications(including RF, microwave, and infrared communications), electroniccommunications, or other such communication means.

The disclosed methods, apparatus, and systems should not be construed aslimiting in any way. Instead, the present disclosure is directed towardall novel and nonobvious features and aspects of the various disclosedembodiments, alone and in various combinations and sub combinations withone another. The disclosed methods, apparatus, and systems are notlimited to any specific aspect or feature or combination thereof, nor dothe disclosed embodiments require that any one or more specificadvantages be present or problems be solved.

The technologies from any example can be combined with the technologiesdescribed in any one or more of the other examples. In view of the manypossible embodiments to which the principles of the disclosed technologymay be applied, it should be recognized that the illustrated embodimentsare examples of the disclosed technology and should not be taken as alimitation on the scope of the disclosed technology.

1-20. (canceled)
 21. A computing device comprising: a processing unit;and memory; the processing unit configured to execute computerinstructions performing operations for co streaming within a liveinteractive video game streaming service, the operations comprising:automatically determining that two or more broadcasters are eligible toco-stream; based at least in part on automatic acceptance by the two ormore broadcasters, creating a co-stream comprising each of the two ormore broadcasters' video game streams; and streaming the co-stream to aplurality of viewer devices for providing to viewers that were currentlywatching broadcast channels of any of the two or more broadcastersbefore the co-stream was created.
 22. The computing device of claim 21,wherein the co-stream is automatically accepted by the two or morebroadcasters without broadcaster intervention.
 23. The computing deviceof claim 21, wherein the automatic acceptance by the two or morebroadcasters is controlled by user-configurable settings, and whereinthe user-configurable settings indicate automatic acceptance of theco-stream without broadcaster intervention.
 24. The computing device ofclaim 21, wherein automatically determining that the two or morebroadcasters are eligible to co-stream comprises determining that thetwo or more broadcasters are in a same party.
 25. The computing deviceof claim 21, wherein each of the two or more broadcaster's video gamestreams are independently included in the co-stream, and wherein each ofthe two or more broadcaster's video game streams are individually savedby the live interactive video game streaming service and individuallyaccessible by the plurality of viewer devices.
 26. The computing deviceof claim 21, the operations further comprising: generating combined chatinformation from each of the viewers of the two or more broadcasters;and sending the combined chat information to the plurality of viewerdevices.
 27. The computing device of claim 26, wherein moderatorsassociated with any of the two or more broadcasters before the co-streamwas created are permitted to moderate viewers across all broadcasters ofthe co-stream while the co-stream is active.
 28. A computing devicecomprising: a processing unit; and memory; the processing unitconfigured to execute computer instructions performing operations for costreaming within a live interactive video game streaming service, theoperations comprising: based at least in part on joining of two or morebroadcasters, creating a co-stream comprising each of the two or morebroadcasters' video game streams; and providing an option to stream theco-stream to a plurality of viewer devices to viewers that werecurrently watching broadcast channels of any of the two or morebroadcasters before the co-stream was created.
 29. The computing deviceof claim 28, wherein the co-stream is automatically joined by the two ormore broadcasters without broadcaster intervention.
 30. The computingdevice of claim 28, wherein the joining by the two or more broadcastersis controlled by user-configurable settings, and wherein theuser-configurable settings indicate automatic joining of the co-streamwithout broadcaster intervention.
 31. The computing device of claim 28,the operations further comprising: automatically determining that two ormore broadcasters are eligible to co-stream.
 32. The computing device ofclaim 28, wherein automatically determining that the two or morebroadcasters are eligible to co-stream comprises determining that thetwo or more broadcasters are in a same party.
 33. The computing deviceof claim 28, wherein each of the two or more broadcaster's video gamestreams are independently included in the co-stream, and wherein each ofthe two or more broadcaster's video game streams are individually savedby the live interactive video game streaming service and individuallyaccessible by the plurality of viewer devices.
 34. The computing deviceof claim 28, the operations further comprising: generating combined chatinformation from each of the viewers of the two or more broadcasters;and sending the combined chat information to the plurality of viewerdevices.
 35. The computing device of claim 34, wherein moderatorsassociated with any of the two or more broadcasters before the co-streamwas created are permitted to moderate viewers across all broadcasters ofthe co-stream while the co-stream is active.
 36. A method, implementedby a computing device, for co-streaming within a live interactive videogame streaming service, the method comprising: by the live interactivevideo game streaming service: based at least in part on joining of twoor more broadcasters, creating a co-stream comprising each of the two ormore broadcasters' video game streams; and streaming the co-stream, asindependently accessible streams, to a plurality of viewer devices toviewers that were currently watching broadcast channels of any of thetwo or more broadcasters before the co-stream was created.
 37. Themethod of claim 36, wherein the co-stream is automatically joined by thetwo or more broadcasters without broadcaster intervention.
 38. Themethod of claim 36, wherein the joining by the two or more broadcastersis controlled by user-configurable settings, and wherein theuser-configurable settings indicate automatic joining of the co-streamwithout broadcaster intervention.
 39. The method of claim 36, furthercomprising: automatically determining that two or more broadcasters areeligible to co-stream.
 40. The method of claim 36, wherein theindependently accessible streams of the co-stream are independentlydisplayable by the viewers at the plurality of viewer devices.